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INFRARED  IMAGERY  RECOGNITION  [AND]  PROCESSING 


Abstract:  This  document  addresses  in  simple  terms  the  flow  of  information  in  infrared 
imagery  recognition  and  processing,  and  proposes  using  knowledge-based  expert  systems  and 
advanced  neural  networks  to  bring  about  a  development  direction  for  automated,  intelligent 
imagery  recognition  and  processing.  This  document  also  presents  a  kind  of  effective  missile- 
•  borne  infrared  imagery  detection,  tracking,  and  extrapolation  method. 
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Infrared  imagery  recognition  processing  systems  are 
indispensable  component  parts  of  infrared  imagery  guidance 
heads.  They  belong  to  one  of  the  key  technologies  in 
precision  guidance  systems.  In  modern  defense  technology/ 
they  possess  an  important  place,  and  they  are  one  of  the 
signs  of  weapon  system  automation,  conversion  to 
intelligence,  and  even  weapon  system  replacement.  Infrared 
imagery  recognition  processing  research  contents  are 
extremely  abundant — for  example,  infrared  imagery  sequence 
processing,  low  signal  to  noise  ratio  detection,  target 
invariable  characteristic  analysis  and  extraction, 
recognition  of  targets  against  complicated  backgrounds,  real 
time  information  processing  technology  making  special  use  of 
conversion  to  intelligent  target  recognition  associated  with 
expert  systems  and  neural  networks  as  well  as  the  completion 
of  the  contents  of  research  discussed  above,  and  so  on. 


II.  Infrared  Imagery  Recognition  Processing  Flows  as 
Well  as  Ways  of  Actualization 


As  far  as  infrared  imagery  recognition  processing  flows 
are  concerned,  normally,  they  are  divided  into  the  five 
parts  of  preprocessing,  imagery  division,  characteristic 

*  Numbers  in  margins  indicate  foreign  pagination. 

Commas  in  numbers  indicate  decimals. 


extraction,  target  classification,  and  target  tracking.  The 
flow  chart  is  as  shown  in  the  Fig.  below: 


Infrared  Imagery  Recognition  Processing  Flow  Chart  (1) 
Imagery  Input  (2)  Preprocessing  (3)  Imagery  Division 
(4)  Characteristics  Extraction  (5)  Target  Classification 
( 6 )  Target  Tracking 

1.  Preprocessing 

Preprocessing  primarily  includes  wave  filtering  and 
multiple  target  detection.  As  far  as  fighting  under  the 
conditions  of  bad  battlefield  environments  is  concerned,  due 
to  target  infrared  radiation  being  relatively  weak,  it  is 
necessary  to  filter  out  noise  and  environmental  interference 
in  the  infrared  imagery,  increasing  the  effectiveness  and 
reliability  of  target  detection  and  recognition,  and,  in 
conjunction  with  this,  creating  good  conditions  for  imagery 
division.  At  the  present  time,  what  is  most  commonly  seen 
is  the  option  to  use  false  middle  value  wave  filters.  It 
possesses  the  advantages  of  restraining  noise  and  increasing 
signal.  Moreover,  it  is  easy  to  use  hardware  and  software 
to  realize. 

Multiple  target  detection  is  a  prerequisite  for  target 
recognition.  From  time  order  imagery,  a  rapid  decision  is 
made  on  whether  or  not  targets  exist  in  the  area.  In 
conjunction  with  this,  initial  fixing  is  realized  on  targets 
that  appear.  High  brightness  points  in  detection  imagery 
act  as  target  candidates.  In  conjunction  with  this,  through 
open  windows,  it  was  possible  to  very,  very  greatly  reduce 
the  amounts  of  calculations. 
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2.  Imagery  Division 
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Imagery  division  is  nothing  else  than  in  accordance 
with  certain  characteristics— taking  imagery  space  and 
dividing  it  into  a  number  of  regions.  Imagery 
characteristics  within  the  same  region  are  the  same. 
Imagery  characteristics  of  different  regions  are  variously 
different.  The  objective  is  to  take  the  forms  of  all 
"possible  targets"  within  windows  with  target  points  as 
centers  and  divide  them  out- 


Opting  for  the  use  of  imagery  relaxation  division 
methods  based  on  models,  before  extracting  target 
geometrical  characteristics,  they  will  be  precisely  cut  away 
from  the  background  and  extracted.  Threshold  value 
processing  is  carried  out,  and,  in  conjunction  with  that, 
they  are  converted  to  dual  values.  Moreover,  division 
threshold  values  are  capable  of  automatic  selection  in  order 
to  adapt  external  boundary  condition  changes  to  the 
influences  of  infrared  imagery.  This  type  of  gate  limit 
characteristic  method  is  only  capable  of  obtaining  partial, 
incomplete  target  information.  The  reason  is  that,  during 
imagery  division  processes,  there  will  sometimes  be  produced 
target  edge  "cracks"  or  the  appearance  of  pit  phenomena.  It 
is  difficult  to  extract  a  closed  target  profile  among 
background  noise.  Frequently,  it  is  also  necessary  to  opt 
for  the  use  of  methods  associated  with  the  combining  and 
division  of  target  region  characteristics  and  edge 
characteristics  in  order  to  synthesize  a  complete  target. 

3 .  Characteristics  Extraction 

Target  characteristic  extraction  plays  an  important 
role  in  imagery  recognition.  As  a  result,  it  is  a  key 
technology  for  real  time  imagery  tracking  systems.  It  does 
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not  only  influence  the  accuracy  of  recognition  and 
classification.  It,  moreover,  also  influences  the 
algorithms  and  the  speed  of  calculations.  Target 
characteristics  are  nothing  else  than  various  types  of 
parameters  reflecting  target  attributes.  They  are  digitized 
and  become  characteristics  data  which  computers  are  capable 
of  receiving  and  are  used  in  order  to  carry  out 
classification,  description,  and  tracking  of  recognized 
objects.  Characteristics  which  can  be  selected  are  very 
numerous — for  example,  structure  characteristics  (target  two 
dimensional  projection  length,  width,  circumference,  area, 
and  so  on),  statistical  characteristics  (target  mean  sguare 
deviation,  mean  values,  and  so  on),  and  spacial 
characteristics  (target  position,  speed,  distance,  and  so 
on) .  The  principles  for  selecting  characteristic  digits 
should  be  based  on  the  requirements  of  effective 
recognition.  As  much  as  possible,  selection  should  be  made 
of  relatively  few  numbers  in  order  to  reduce  the  complexity 
of  target  classification. 

4.  Target  Classification 


Target  classification  includes  "target 
preclassification  processing"  (that  is,  after  going  through 
preprocessing,  imagery  division,  and  characteristics 
extraction,  the  carrying  out  of  marking  on  targets  existing 
in  imagery)  and,  on  the  basis  of  selection  characteristics, 
the  execution  on  the  many  targets,  of  tracking,  side 
recognition,  and  final  recognition  as  real  targets  or  main 


targets . 

Target  classification  methods  are  very  numerous.  Each 
has  special  characteristics.  For  example,  methods  based  on 
characteristic  vector  matching  possess  the  advantage  of  high 


speed  processing.  Laminar  Manhedun  (phonetic)  minimum 
distance  methods  possess  relatively  high  recognition 


probabilities . 


"Neural  network  like"  methods  associated 


t 


* 
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with  multidimensional  gradients  allow  one  to  have  relatively 
large  characteristic  error  ranges. 

5 .  Target  Tracking 

Before  real  targets  are  recognized,  multiple  target 
tracking  is  carried  out  of  target  objects  in  the  imagery. 

At  the  same  time,  movement  characteristics  are  supplied 
constantly  to  classification  devices.  Single  target 
tracking  only  requires  supplying  target  angular  deviations . 
In  conjunction  with  that,  error  signals  are  inputed  into 
control  systems  to  carry  out  corrections,  guaranteeing  that 
the  center  of  targets  are  placed  at  the  center  of  infrared 
fields  of  view.  Multiple  target  tracking  should  track 
multiple  targets,  recording  their  positions  in  the  field  of 
view.  Moreover,  as  far  as  the  carrying  out  of  tracking  of 
objects  in  groups  is  concerned,  real  time  information  should 
also  be  given  on  their  angular  positions.  It  is  possible  to 
opt  for  the  use  of  characteristic — correlation  compensation 
tracking  methods  to  carry  out  real  time  position  fixing  of 
various  individual  targets  from  sequenced  imagery  in  each 
frame.  After  real  targets  are  recognized,  they  are 
transformed  into  single  target,  self "adapting  tracking 
window  tracking  configurations.  In  conjunction  with  this, 
through  algorithms,  precise  determinations  are  made  of  the 
positions  and  speeds.  Making  use  of  Kaerman  (phonetic) 
target  tracking  algorithms  based  on  gradient  measurements, 
j_terative  substitution  approximate  target  center  of  mass 
tracking  algorithm  methods,  target  tracking  algorithms  based 
on  center  of  gravity  vectors,  target  tracking  algorithms 
after  inserting  FFT,  as  well  as  automatically  changed  window 
dimension  algorithms,  and  so  on,  tracking  precision  is 
0xtremely  high.  They  are  capable  in  all  cases  of  reaching 
sub  image  element  orders  of  magnitude. 
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6.  Directions  in  the  Development  of  Infrared  Imagery 
Processing  Technology 

Infrared  imagery  recognition  processing  is  an 
interesting  new  technology  that  began  to  develop  over  the 
last  twelve  years.  With  regard  to  the  development  of 
precision  guidance  technology  and  space  interception 
technology,  they  possess  key  significance.  At  the  present 
time,  it  has  already  become  a  hot  topic  in  the  midst  of 
great  efforts  at  development  and  research  inside  China  and 
abroad.  Automatic  target  recognition  technology  is  one  of 
the  key  technologies  which  the  U.S.  Defense  Department 
listed  as  main  points  to  be  developed.  As  far  as  the 
"Youchu  (phonetic)  [Chinese  means  a  young  domestic  animal]" 
infrared  imagery  guided  missile  which  the  U.S.  already 
possesses  and  the  SLAM,  which  relies  mainly  on  artificial 
participation  for  recognition,  are  concerned — with 
automation  and  conversion  to  intelligent  imagery  recognition 
technology — it  can  be  expected  to  achieve  practical 
applications  by  the  end  of  this  century. 

Imagery  recognition  processing  systems  turned 
intelligent  should  possess  such  functions  as  imagery 
understanding,  autonomous  planning,  formulation  of  knowledge 
and  reasoning,  self-adjusting  operation  and  control, 
mechanical  learning,  expert  systems,  artificial 
intelligence,  and  so  on.  At  the  present  time,  development 
trends  are  neural  network  expert  systems .  Due  to  neural 
network  methods,  it  is  possible  to  make  effective  use  of 
them  in  imagery  division.  Target  recognition  capabilities 
are  strong.  Also,  due  to  the  fact  that  they  have  the 
characteristics  of  possessing  learning  capabilities  and 
permitting  large  errors,  as  a  result,  self -adapting 
capabilities  are  relatively  strong.  They  are  even  more 
suitable  for  use  with  environmental  changes  and  have  already 


/52 


6 


become  important  points  in  the  direction  of  development. 

At  the  present  time,  methods  opting  for  the  use  of 
imagery  recognition  processing  are  relatively  numerous. 
Generally,  they  are  all  capable  of  achieving  relatively  good 
target  detection  and  classification  results.  The  direction 
for  further  research  is  how  to  raise  operating  speeds . 
Actualization  is  easy.  In  order  to  satisfy  engineering 
applications,  EKF  neural  networks,  which  are  in  the  process 
of  being  explored,  have  hopes  of  turning  into  the  direction 
for  the  main  attack. 

III.  Onboard  Infrared  Imagery  Recognition  Processing 
Methods 


There  are  relatively  large  differences  between  infrared 
imagery  recognition  processing  onboard  missiles  or  shells 
and  ordinary  imagery  processing  devices.  First  of  all,  they 
are  a  fully  automatic  imagery  processing  system.  Second, 
they  require  that  algorithms  be  simple  and  reliable.  In  the 
entire  recognition  processing  process,  taking  imagery  and 
matching  tracking  is  the  main  thing.  Target  recognition  and 
decisions  are  secondary.  This  is  because,  entering  into  the 
target  tracking  phase,  initial  infrared  information 
processing  has  already  taken  targets  and  drawn  them  into 

infrared  fields  of  view. 

1.  Infrared  Imagery  Recognition  Processing 

From  infrared  imagery  obtained  by  onboard  sensors, 
generally  speaking,  relatively  strong  target  data  is  already 
included  (introduced  by  initial  guidance).  The  elimination 
of  noise  and  the  extraction  of  targets  from  the  imagery  in 
question  is  the  first  operational  step  associated  with 
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infrared  imagery  recognition  processing. 


Imagery  obtained  from  infrared  sensors  is  capable  of 
being  represented  using  the  two  dimensional  function  f(x,y). 
In  conjunction  with  this,  it  is  possible  to  divide  it  into 
two  parts 


/(x^y)  =  f,(.x^y)  +  fe(xyy) 


In  this,  fs(x,y)  and  fo(x,y)  are,  respectively, 
background  and  target  image  functions .  After  taking  the 
imagery  functions  in  question  and  digitizing  them,  they  are: 


fCxi^yj)  =  f  ,<.Xi^yj}  +  f  o(.Xi^yj) 


Due  to  background  functions  generally  being  capable  of 
using  a  normal  distribution  N  =  to  represent  the 

statistical  characteristic  of  fs(xi,yj),  by  comparison, 
target  signal  ratios  are  relatively  strong.  Moreover,  when 
it  is  a  target  image  fo(xi,  yi)  possessing  a  fixed  form,  it 
can  be  represented  using  the  matrix 
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f  o(.Xi^yj'> 


I  0 . 

j  :  X  0  X 
=1  :  X  X  X 

j  0  0  X  0 
"  0 . 

■  JT 


\M 


0  // 


(1) 


In  form  (1)  x  stands  for  non  zero  values.  The 
distribution  of  non  zero  values  associated  with  the  matrix 
is  the  target  planar  projected  accumulation  on  detectors. 

The  matrix  of  background  function  fs(xi,yj)  is 
expressed  in  the  form: 


^5*  0  0)  5*  0  1 . S'  o/V  > 

yi)  = 

\ 

\ 

\ 

\ 

\ 

\ 

^  oiV . Snjif 

(2) 
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Among  these,  each  element  Sij  is  a  distribution  N(/i,a). 
Due  to  backgrounds  being  relatively  clean,  it  is,  therefore, 
possible  to  postulate  the  expected  values  /it, a  being 
relatively  small  compared  to  targets. 


/53 

With  regard  to  the  processing  of  this  type  of  infrared 
imagery,  using  mathematical  statistics  methods  and  putting 
them  together  with  imagery  processing  is  one  type  of 
appropriate  consideration  (Mathematical  statistics 
treatments  are  capable  of  lowering  amounts  of  calculations 
in  a  large  way.  In  conjunction  with  this,  guick  responses 
are  made  to  algorithms . ) 

Because  what  detection  devices  detect  is  the  sum  of 
target  imagery  and  background  imagery,  and  also,  because 
average  degrees  of  target  contrast  are  far  larger  than 
average  degrees  of  background  contrast,  it  is,  therefore, 
desirable  to  opt  for  the  use  of  imagery  contrast  checks  and 
separations  and,  in  conjunction  with  that,  recognize 
targets . 

Average  degrees  of  contrast  within  imagery  frames  can 
be  expressed  as: 


M  N 
1=1  7  =  1 


(3) 
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Using  multiples  of  average  contrasts  f  as  gate  limits, 
L  individual  imagery  units  with  relatively  strong  signals 
are  screened  out  from  frames  of  imagery . 


<//?>  /?  =  0,1,2, . L-l 


(4) 


The  function  making  measurements  for  the  screening  out 
of  {fR}  individual  image  elements  is: 


=  ,/,•>{>/ =  0,1 . L-l 


(5) 


Note:  M<fi,fj>  is  {fR}  position  measure. 


Using  C{fR}  measures  to  act  as  target  shape  correlation 
checks,  targets  are  finally  recognized. 
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what  needs  to  be  pointed  out  is  that  C{fR}  is  position 
measures  associated  with  L  individual  samples  screened  out. 
In  reality,  it  also  plays  the  role  of  screening  out  noise 
because  image  elements  which  do  not  correlate  with  target 
shapes  will  be  eliminated. 

Going  through  methods  above  to  check  false  alarm  rates 
in  target  recognition,  they  are  on  the  order  of 
approximately  10-3  sec.  Due  to  the  fact  that,  when 
infrared  sensors  operate,  the  target  forms  which  detectors 
detect  are  relatively  large.  Imagery  resolution,  therefore, 
can  be  a  bit  low,  facilitating  simplified  algorithms  and 
lowering  costs. 

Based  on  our  calculations  and  experiments,  the  general 
resolution  rate  for  imagery  sensors  is  32x32  which  is 
relatively  appropriate. 

2.  Infrared  Imagery  Tracking  Processing 

The  main  task  of  infrared  imagery  tracking  processing 
is  to  switch  over  to  tracking  after  targets  have  been 
identified.  Normally,  option  is  made  for  the  use  of 
self-adjusting  tracking. 

a.  Contrast  Self-Adjusting  Tracking 

Let  T  be  contrast  operator 


r=C/(xi,  yj))  =  {A}L  (6) 

In  this,  A  is  gate  limit  for  one  contrast  test. 

Contrast  self-  adjustment  indicates  that  the  next  gate  limit 
value  is  capable  of  fluctuating  above  or  below  value  A  in 
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accordance  with  a  certain  coefficient.  Using  this  method, 
it  is  possible  to  improve  tracking  target  drift  caused  by 
changes  in  contrast. 

b.  Image  Criteria  Ordered  Penetration  Correlation 
Tracking 

After  checking  targets  with  image  recognition 
processing,  the  geometrical  center  of  the  L  individual  image 
elements  screened  out  are  calculated  out  as  well  as  second 
order  moments  (target  centers  of  gravity)  as  imagery 
criterion  St  .  In  conjunction  with  this,  on  imagery 
criteria  St  for  the  imagery  frame  in  question  as  well  as  the 
previous  imagery  frame  St-1  ,  sequenced  penetration 
correlation  comparisons  are  carried  out: 


St-S,.i<e  t  =  l,2, . n 


(7) 


£  is  tracking  deviation.  It  is  precisely  determined  by 
system  design.  When  equation  (7)  is  inadequate,  tracking 
switches  over  into  new  exploration  and  recognition. 


Sequenced  penetration  correlation  methods  are  one  hot 
point  in  recognition  and  tracking  technology  inside  China 
and  abroad  in  the  last  few  years.  When  system  design  is 
relatively  good,  the  amounts  of  calculation  and  processing 
are  relatively  small.  Recognition  and  tracking  precision, 
in  all  cases,  are  capable  of  achieving  increases.  At  the 
same  time,  sequenced  penetration  method  convergence 
characteristics  are  relatively  good.  However,  it  is 
necessary  to  correctly  select  deviation  amounts  .  In 
conjunction  with  this,  with  regard  to  target  statistical 
parameters  (for  example,  distribution  functions, 
distribution  likelihood  ratio  functions,  and  so  on),  there 
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must  be  a  relatively  accurate  estimate. 


3.  Memory  Extrapolation  Tracking 

Memory  extrapolation  tracking  technology  is  a  type  of 
specialized  emergency  tracking  measure.  At  the  present 
time,  a  good  number  of  weapons  systems  outside  China  possess 
this  type  of  function.  With  main  attention  on  times  when 
targets  are  normally  tracked,  targets  are  suddenly  blocked 
and  disappear  for  short  periods.  After  a  few  seconds,  they 
again  normally  reappear.  In  accordance  with  normal  tracking 
treatment,  one  will  then  have  the  appearance  of  target  loss, 
system  disorder,  and  bungled  combat  opportunities.  Because 
of  this,  memory  extrapolation  tracking  becomes  very 
necessary. 

Memory  extrapolation  is  nothing  else  than  making  use  of 
target  information  from  the  previous  frame  and  this  frame. 

In  conjunction  with  this,  storage  memory  is  added.  After 
that,  use  is  made  of  predictive  algorithms  to  forecast  the 
target  parameters  in  the  next  frame.  The  rest  may  be 
deduced  by  analogy.  Extrapolation  algorithms  are  very 
numerous .  They  have  various  advantages  and 

disadvantages — for  example,  approximation  /54  algorithms  are 
simple  and  fast.  However,  accuracy  is  relatively  bad. 
Kaerman  (phonetic)  wave  filter  methods  are  advanced,  but 
computations  are  complicated.  We  opt  for  the  use  of 
"differential  linear  drafting  extrapolation"  methods.  The 
basic  important  points  are  that  imagery  is  reduced  from 
three  dimensions  to  two  dimensions,  inertial  limited  non 
stable  processes,  and  blocking  of  previous  tracking  data 
(target  center  positions)  which  have  already  entered  into 
processors  (that  is,  memory  algorithms).  Once  targets  are 
lost,  it  is  necessary,  by  processors  making  use  of  previous 
target  information  (existing  memory),  to  predict  the 
target's  next  position  on  the  basis  of  differential  linear 
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drafting  extrapolation.  Relying  on  this,  they  cycle  right 
through  to  the  target ’ s  appearance .  A  summary  of  the 
algorithm  is  as  follows; 


'  XUi)  yiti)  ,-  =  0,1,2 . -JV-l. 


for  a  target  at  instant  ti  and  coordinate  x,y. 
Doing  differential  differences, 


^yi-yiti  +  l)  - y(.ti) 

I  ~  0, 1 ,2, . 


4 


Doing  slope  drafts  (precisely  specifying  coefficient  K) 
-expected  average  method 


AT-l 

iJ'=E  ^Xi/N 


1=0 


C8) 
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least  square  method 

On  the  foundation  of  expected  average  methods,  minimum 
square  approximations  were  done  on  K  to  solve  for  basic 
values 


;v-i 


1=0 


N-l 


i-O 


for  the  condition 


Making  K  corrections 


N-l 

E  (^Xi-aX)WN  =  E, 

1=0 

for  the  condition  a<^i 
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Respectively  comparing  /  use  is 

made  of  recurrence  methods  to  make  values  minimum,  a^^ 
which  are  obtained  are  minimum  square  approximate  slope 
drafts  (  that  is,  draft  quality  is  the  best). 

l^yi  can  also  be  obtained  in  an  analogous  way. 

Summarizing  what  was  discussed  above,  on  board  infrared 
imagery  recognition  tracking  is  divided  into  two  parts •  One 
is  target  recognition  extraction  which  primarily  opts  for 
the  use  of  imagery  processing  and  statistical  decision 
making  methods  combined  together  for  realization.  The 
second  is  tracking  which  primarily  opts  for  the  use  of 
self-adjusting  tracking  modes.  When  tracking  requirements 
low,  selection  was  made  of  self—  adjusting  contrast 
tracking  which  is  economical  in  practical  use.  When 
tracking  requirements  are  high,  target  shape  self-adjusting 
tracking  is  selected  for  use  (matching  wave  filter) . 
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